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Abstract 


This  paper  analytically  compares  the  performance  of  the  SRPT  (Shortest- Remaining- 
Processing-Time)  and  PS  (Processor-Sharing)  scheduling  policies.  SRPT  scheduling  has 
long  been  criticized  for  treating  large  jobs  unfairly,  whereas  PS  scheduling  is  by  definition 
fair.  We  evaluate  SRPT  and  PS  under  conditions  where  the  unfairness  under  SRPT  is  be¬ 
lieved  to  be  most  apparent:  system  overload.  Specifically,  we  consider  an  single  server  with 
an  alternating  ON/OFF  arrival  process.  During  the  ON  periods  the  load  at  the  server  ex¬ 
ceeds  1.  During  the  OFF  periods  t  he  load  at  the  server  is  0.  We  derive  expressions  for  mean 
response  time  as  a  function  of  job  size  under  PS  and  SRPT,  for  general  job  size  distributions. 
In  comparing  these  expressions,  we  find  that  for  our  ON/OFF  model: 

1.  The  mean  response  time  under  SRPT  scheduling  is  far  lower  than  under  PS  scheduling. 

2.  When  the  job  size  distribution  is  exponential,  the  biggest  jobs  may  have  higher  mean 
response  time  under  SRPT  scheduling  as  compared  with  PS  scheduling.  However, 
when  the  job  size  distribution  is  heavy-tailed  all  jobs,  including  the  very  largest  job, 
have  lower  (or  only  marginally  higher)  mean  response  times  under  SRPT  scheduling 
as  compared  with  PS  scheduling.  Heavy-tailed  workloads  are  important  because  they 
arise  naturally  in  many  empirical  computer  workloads. 


1  Introduction 


In  computer  systems  today,  when  multiple  jobs  contend  for  a  single  resource  (e.g.  CPU  or 
bandwidth),  the  policy  used  for  scheduling  the  jobs  most  closely  resembles  Processor-Sharing 
(PS).  That  is,  the  desired  resource  is  time-shared  among  the  contending  jobs,  with  each  job 
in  turn  receiving  a  small  quantum  of  service. 

It  is  well-known  that  the  Shortest-Remaining-Processing-Time-First  (SRPT)  scheduling 
policy  yields  better  mean  response  time  than  PS  scheduling,  however,  applications  have 
shied  away  from  using  this  policy  for  fear  that  SRPT  “starves”  big  jobs  [2,  14,  15,  13].  The 
fear  is  that  the  response  times  of  the  big  jobs  will  be  much  higher  under  SRPT  scheduling 
than  under  PS  scheduling. 

In  a  recent  paper  [1],  we  proved  that  in  the  case  of  a  single  M/G/1  queue  where  the  job 
size  (processing  requirement,  or  service  requirement)  distribution  is  heavj^-tailed,  this  fear 
is  unsubstantiated.  We  showed  that,  provided  the  system  load  p  is  such  that  p  <  I  and  p 
is  not  too  close  to  1,  and  provided  the  job  size  distribution  is  heavy-tailed,  then  all  jobs, 
including  the  very  largest  job  have  lower  mean  response  time  under  SRPT  scheduling  than 
under  PS  scheduling  (and  the  overall  mean  response  time  is  far  lower  under  SRPT  scheduling 
as  compared  with  PS  scheduling).  The  above  analysis  required  that  p  not  be  too  close  to  1 
(e.g.  p  <  .99),  so  as  to  allow  large  jobs  a  turn  to  run.  The  above  analysis  was  substantiated 
further  by  a  second  paper  [6],  where  we  implemented  SRPT  scheduling  in  a  Web  server,  and 
compared  performance  of  our  SRPT-modified  Web  server  with  the  original  Web  server  in 
the  case  of  p  <  .95  under  trace-based  Web  workloads  (which  are  heavy-tailed).  (Web  servers 
are  a  particularly  good  application  for  SRPT  scheduling,  since  the  service  requirement  of  a 
Web  request  is  proportional  to  the  size  of  the  file  being  requested,  which  is  known)  [6]. 

However,  in  real  systems,  such  as  Web  servers,  we  can  not  be  guaranteed  that  the  load 
will  always  stay  below  1.  In  fact  it  is  much  more  common  that  the  load  bounces  around, 
sometimes  exceeding  1  and  sometimes  dropping  to  0.  Starvation  under  SRPT  would  appear 
to  be  more  of  a  problem  under  overloaded  conditions.  The  intuition  commonly  given  is: 

A  big  job  arrives.  Immediately  thereafter  a  whole  bunch  of  small  jobs  arrive,  and 
the  big  job  never  gets  to  run  under  SRPT  [2,  4,  H,  15,  13]. 

The  above  statement  would  be  obvious  if  the  system  remained  in  overload  forever.  How¬ 
ever,  it’s  not  clear  what  exactlj'  happens  if  the  system  alternates  between  overloaded  and 
underloaded  conditions  (with  mean  system  load  under  1).  Do  big  jobs  necessarily  do  better 
under  PS  as  compared  with  SRPT? 

In  this  paper,  we  therefore  consider  an  ON/OFF  load  model,  where  the  arrival  rate 
alternates  between  overload  and  zero  load.  The  exact  model  is  described  in  Section  2.  For 
analytical  tractability,  we  assume  that  the  duration  of  the  ON  and  OFF  periods  approach 
infinity.  We  assume  a  general  job  size  distribution 

Under  the  above  model  we  derive  the  mean  response  time  as  a  function  of  the  job  size 
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under  PS  scheduling  and  under  SRPT  scheduling.  We  also  derive  several  other  interesting 
performance  metrics,  including  the  fraction  of  arriving  jobs  which  are  still  in  the  system  as 
a  function  of  time,  under  SRPT  and  under  PS  scheduling.  This  latter  metric  has  practical 
importance  since  it  factors  heavily  into  system  overhead  (e.g.  context  switching  time  and 
number  of  state  buffers  which  must  be  maintained).  We  then  evaluate  these  formulas  in  the 
case  of  a  few  specific  job  size  distributions. 

We  find  that  when  the  job  siz.e  distribution  is  cxponenfiaL  SRPT  improves  over  PS  with 
respect  to  mean  response  time  and  number  of  jobs  in  the  system  by  at  least  a  factor  of 
4.  However,  this  improvement  comes  at  a  price:  the  expected  response  time  for  large  jobs 
under  SRPT  can  be  about  2-3  times  liigher  than  under  PS.  This  "relative  starvation’'  of  the 
large  jobs  turns  out  to  depend  on  the  degree  of  overload  during  the  ON  period.  The  relative 
starvation  of  the  big  jobs  under  SRP'f  as  compared  with  PS  turns  out  to  be  worst  when  the 
load  during  the  ON  period  is  just  slightly  above  1. 

However,  when  the  job  size  distribution  is  heavy-tailed,  which  is  characteristic  of  many 
computer  workloads  [11.  8,  5,  9.  12].  the  story  changes.  Under  a  Bounded  Pareto  job  size 
distribution  with  o-parameler  of  l.o.  we  find  that  SRPT  improves  over  PS  with  respect  to 
the  mean  response  times  of  jobs  and  the  number  of  jobs  in  the  system  by  over  an  order  of 
magnitude.  These  relative  im])rovements  of  SRPT  over  PS  are  in  fact  greatest  when  the  load 
during  the  ON  period  is  not  too  high.  Moreover,  this  improvement  in  mean  performance 
does  not  come  at  the  cost  of  hurting  the  j)erformance  of  large  jobs.  In  fact,  even  the  very 
largest  job  has  ecpial  (or  only  marginally  higher)  response  time  under  SRPT  as  compared 
with  PS. 

In  Section  5  we  discuss  the  above  results  in  detail  and  provide  intuition  for  why  they 
hold.  The  above  results  are  encouraging  with  respect  to  the  potential  real-world  applicability 
of  SRPT  scheduling. 

2  Problem  Formulation  and  Notation 


Throughout  this  paper  we  will  be  assuming  an  M/G/1  queue.  Job  sizes  will  be  denoted  by 
the  random  variable  5.  The  job  sizes  will  be  assumed  to  be  independent  and  identically 
distributed  with  c.d.f.  F(x)  and  p.d.f.  f{x). 

The  arrival  process  will  consist  of  alternating  ON/OFF  periods.  During  the  ON  period 
(also  called  the  “high  load"  period),  jobs  arrive  with  mean  arrival  rate  A/^  and  create  a  load 
of  /9/j,  >  1.  The  high  load  period  has  fixed  duration  t^.  During  the  OFF  period  (also  called 
the  “low  load”  period)  jobs  arrive  with  mean  arrival  rate  A/  and  create  a  load  of  pi  =  0.  The 
low  load  period  has  fixed  duration  //. 

Let  p  denote  the  average  system  load.  Thus 
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We  will  always  assume  that  the  average  load  p  <  1, 

Our  model  assumes  alternating  ON  and  OFF  periods.  During  the  ON  period,  jobs  build 
up  in  the  system  as  a  function  of  time.  We  will  derive  an  expression  for  this  buildup.  At  the 
start  of  the  OFF  period,  there  is  an  accumulation  of  jobs  which  we  refer  to  as  ‘‘the  bag  of 
jobs.”  We  will  compute  a  distribution,  on  the  remaining  processing  requirements  of  jobs 
in  the  bag  at  the  start  of  the  OFF  period.  The  mean  of  this  distribution  will  be 

If  we  assume  that  th  and  ti  are  finite,  then  we  cannot  guarantee  that  with  probability 
1  the  number  of  jobs  at  the  beginning  of  every  ON  period  is  zero.  Furthermore,  deriving 
the  system  state  under  finite  th  and  ti  is  extremely  difficult.  Thus  we  need  to  assume  that 
th  oo  and  oc.  In  this  case  it  is  easy  to  see  by  the  Central  Limit  Theorem,  that  when 
the  average  load,  />  <  1,  the  number  of  jobs  at  the  beginning  of  every  ON  period  is  zero  with 
probability  1. 

Some  additional  notation:  The  number  of  jobs  in  the  system  after  t  time  into  the  high 
load  period  will  be  denoted  by  Nh{t),  Likewise  the  number  of  jobs  in  the  sj^stem  after  t  time 
into  the  low  load  period  will  be  denoted  by  Ni{t).  Lastly,  in  analyzing  SRPT  scheduling,  it 
will  be  useful  to  denote  b}^  p{y)  the  load  made  up  of  jobs  of  size  <  y. 


3  Previous  Work 


We  have  not  found  any  literature  discussing  this  problem  prior  to  1994.  In  1994  Robert  and 
Jean-Marie  [10]  analyzed  the  PS  scheduling  policy  in  the  case  of  overload  (not  an  ON/OFF 
process)  and  derived  the  number  of  jobs  in  the  system  as  a  function  of  t  in  the  limit  where 
i  oo.  They  also  derived  the  limiting  distribution  of  remaining  times  on  jobs  in  the  system 
in  the  case  of  t  oo.  We  use  this  result  as  a  step  within  our  proof  of  Theorem  1. 

In  1997  Chen,  Kella,  and  Weiss  [3]  analyzed  the  number  of  jobs  in  the  system  under  the 
PS  scheduling  policy  in  the  case  of  an  OFF  process  which  begins  with  a  bag  of  jobs.  Our 
goals  differ  from  at  of  Chen  et.  al.  in  that  we  are  concerned  with  response  times  rather  than 
number  of  jobs  in  the  system,  and  also,  we  are  concerned  with  an  ON/OFF  workload.  Our 
analytical  methods  differ  from  those  used  by  Chen  et.  ah. 

We  have  not  found  any  mention  of  SRPT  analysis  under  overload  or  under  our  ON/OFF 
workload  in  the  above  papers  or  in  any  papers  in  the  literature. 


4  Analysis  of  response  times  under  SRPT  and  PS 


In  this  section  we  will  derive  formulas  for  the  expected  response  time  of  a  job  of  size  x  in 
the  M/G/l/PS  and  M/G/l/SRPT  systems  under  the  ON/OFF  load  model. 

We  will  start  with  a  preliminary  lemma,  Lemma  1.  Jean-Marie  proved  this  lemma  about 
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M/G/l/PS  queues  under  overload.  For  completeness  we  give  a  quick  overview  of  their  proof. 


Lemma  1  Consider  an  M/G/l/PS  queue  icith  load  p  >  1  and  average  arrival  rate  A.  Let 
denote  the  number  of  jobs  in  the  system  at  time  t.  Then, 


lim 


Ml 
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where  a  is  the  solution  to  the  following  equation: 
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Proof:  Clearly  lini/_j,x.  <  A,  so  there  exists  h  and  c.  such  that  for  t  >  c,  A'/,(/)  <  bf. 

Consider  a  job  of  size  x,  arriving  at  time  /q  >  c,  then  the  job  departs  at  time  fj,  such 
that 


X  — 


> 


1,  td 


So,  td  <  Thus,  at  time  if  a  job  of  size  x  arriving  at  time  to  >  c  is  present  in  the 

system,  then  certainly  to  >  te~^^' . 

So,  Nh{t)  <  Nk{c)  +  f^'  X{t  ~  te^^^')f{x)dx,  provided  te~^^'''  >  c. 

Choosing  t  large  enough, 


Nh{t)  <  N,{c)  +  Xt{l^Lf{b)) 


where  Lf{b)  is  the  Laplace  transform  of  /  evaluated  at  b.  Thus, 


Ml 

t 


<X{1-Lj{b)) 


Let  us  choose  b  =  limsup^^,^,  tiiiO  ^  Now  observe  that  the  function  h{x)  =  x  —  A(1  — 
Lf{x))  has  a  unique  root  a,  in  the  range  x  >  0.  Moreover,  h{x)  oo  as  x  oo.  Thus,  if 
b  >  then  (1  —  Lf{b))  <  6,  leading  to  a  contradiction.  Thus  b  <  a. 

Similarly  we  can  show  that  lim  inf/^oc>^^^^^  > 

Thus  the  result  follows.  ■ 
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Theorem  1  In  an  M/G/l/PS  system,  under  the  ON/OFF  load  model,  the  expected  response 
time  for  a  job  of  size  x,  E{r(a;)p5}  satisfies 


lim  E{r(»)p5}  (l-e-“) 

th^OO  tfi  2 


where  a  is  defined  as  in  equation  2,  and 


+  [ph  -  1)FpJ.t)  -  i(F,(.T)  -  e— )  -  ^  fiF{x  -  z)e--^dz 

(3) 


and 


Fr{y)  =  - - 


(4) 

(5) 


Proof:  Consider  a  job  of  size  x  which  arrives  into  the  system.  A  few  obvious  observations: 
First,  the  job  must  arrive  somewhere  during  the  ON  period.  Second,  the  job  must  complete 
before  the  next  ON  period  begins.  We  therefore  consider  a  single  ON/OFF  cycle  beginning 
at  time  0. 


We  can  define  some  time,  tj^  during  the  ON  period,  such  that  the  job  of  size  x  will 
complete  before  the  ON  period  is  over  if  and  only  if  the  job  arrives  before  time 


To  compute 
equals  x.  Thus 


observe  that  the  service  received  by  any  job  during  the  period 


ph  dy 

L  Nniy) 


(6) 


Now  by  1,  limf_).co  •  —  ci.  We  will  be  loose  and  write  this  as  Nfi{t) 

case  6  gives  ~  the~^^ ^  or 

lim  —  — 

th-^OO  t}i 

A  rigorous  proof  of  7  is  given  in  the  footnote  below^ . 

Henceforth,  to  keep  the  main  idea  clear,  we  will  not  distinguish  between  lim^^^^oc-  = 
e~^^  and  It  can  be  seen  that  this  will  not  matter,  since  we  will  eventually  be 

concerned  with  the  ratio  of  the  response  time  and  th^  The  arguments  can  be  made  rigorous 
as  in  the  footnote  1. 

^By  Equation  1,  limt_).oo  =  u.  So,  given  an  e  >  0,  3C,  such  that  |  _  q|  e  for  t  >  C, 

Assuming  th  is  large  enough  that  >  C,  we  get  that 

Since  th  -)■  oo,  letting  e  — >  0,  we  get  limt^_i.oo  =  e”"^. 


=  at,  in  which 
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We  consider  two  cases:  the  case  where  the  job  of  size  x  arrives  prior  to  time  and  the 
case  where  the  job  of  size  x  arrives  after  time  tj..  We  will  compute  the  mean  response  time 
in  both  cases. 

First  suppose  that  the  job  of  size  x  arrives  prior  to  time  Let  ta  denote  the  arrival  time 
of  the  job,  and  let  fd  denote  its  departure  time  (where  td  <  tf^).  Then  by  (1),  L/  =  So 

the  response  time  of  the  job  is 


tc.  -  1) 


(8) 


Now  since  th  oo,  we  can  assume  that  is  uniformly  distributed  in  [0,  t.,-],  thus  by  (7) 
and  (8)  we  have  that: 


E{r(.r)|^„  G  [0,^}  =  r  f  (c-  -  l)dy  =  ^  (1  -  e-n  (9) 

Jo  tx  I 

Now  we  approach  the  case  where  G  fids  case,  our  job  of  size  x  does  not 

complete  by  time  The  remaining  size  on  our  job  at  time  //^  will  be  denoted  by  x^  where 


which  evaluates  to 


1 ,  tf, 

X  -  -  log  — 

a  ta 


(10) 


The  remainder  of  this  proof  is  devoted  to  computing  the  time  from  when  the  OFF  period 
begins  until  our  job  completes. 

We  need  some  observations: 

Consider  an  M/G/l/PS  with  load  />  =  0,  starting  at  time  0  with  an  initial  number  of 
jobs  A^/(0),  whose  sizes  are  distributed  with  p.d.f.  /^. 


Observation  1  A  job  of  size  y  canH  depart  until  all  jobs  of  size  <  y  have  departed.  So  at 
the  time  when  the  job  of  size  y  departs,  Ni{0)  •  Fr{y)  jobs  have  df. parted. 


Observation  2  When  a  job  of  size  y  departs,  y  units  of  woj'k  have  been  completed  on  all 
jobs  of  size  >  y. 


From  the  above  two  observations,  it  follows  that  our  job  of  remaining  size  x^  will  complete 
at  time: 

A^/(0)  r  zMz)dz  +  Ni{(y)F;{z)xr 
Jo 
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This  simplifies  to 


NliO)E{Sr}FrAXr) 


(11) 


where  E{5r}  represents  the  mean  remaining  size  on  jobs  at  time  0  and  the  Fr^{xr)  is 
the  equilibrium  distribution  of  Fr  evaluated  at  (i.e.  Fr^{xr)  =  j  Jq"'  Fr(y)dy). 

Now  observe  that  A^/(0)E  {5V}  is  just  the  w^ork  in  the  system  at  time  0,  which  is  equal 
to  (ph  —  l)th*  Thus,  the  time  from  the  start  of  the  OFF  period  until  our  job  of  remaining 
size  Xr  completes  is: 

(Ph  -  l)4^^re(‘'^v) 

So  the  response  time  of  our  job  is: 

tk  -  ta  +  [ph  -  l)thFr^  (12) 

Now,  given  that  ta  is  uniformly  distributed  in  by  (10)  and  (12)  we  have  that, 

E{T{x)\t,.  e  ((4  -  y)  +  iPh  -  l)thF,.^  (.X-  -  bog|))  dy  (13) 

Observe  however  that  we  have  not  yet  derived  That  is,  we  still  need  to  compute  F^, 
the  distribution  on  the  remaining  sizes  of  jobs  at  the  beginning  of  the  OFF  period.  We  do 
this  now: 

We  will  compute  the  number  of  jobs  with  remaining  size  >  y  at  the  moment  that  the 
OFF  period  begins. 

Consider  a  job  of  (original)  size  This  job  will  have  remaining  size  >  y  at  time  iff 
it’s  arrival  time,  ta^  is  such  that  ta  >  (by  the  above  type  of  arguments). 

Thus  the  total  number  of  jobs  of  size  [z^  z  +  dz)  which  have  remaining  size  >  y  at  time 
th  is  the  number  of  jobs  which  arrive  during  and  th^  which  is 


\thfiz){l  -  e-‘^^^-y'>)dz 


(14) 


Therefore  the  total  number  of  jobs  which  have  remaining  size  >  y  at  time  th  is  obtained 
by  integrating  14  over  all  possible  job  sizes  (greater  than  y),  thus, 


/' 


A^fc/(2)(1  -  e 


hdz 


(15) 


To  obtain  F^(^)  observe  that  the  total  number  of  jobs  at  time  th  is  ath*  Thus,  using  15 
the  fraction  of  jobs  with  remaining  size  >  ?/  at  the  start  of  the  OFF  period  will  be  given  by 


_  ^  /yA/(z)(l-e-“M)dz 

Fr{y)  =  - - : - 


(16) 
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Moreover, 


(17) 


To  calculate  E[Sr],  observe  that  E[Sr]  is  equal  to  the  ratio  of  the  total  remaining  work  and 
the  total  number  of  remaining  jobs  at  the  beginning  of  the  OFF  period.  Since  the  total 
remaining  work  at  the  end  of  the  OFF  period  is  {pi^  —  and  the  number  in  system  is  by 
definition  ath, 

E[Sr]  =  (18) 


Thus,  we  get 


Finally,  since 


F,,(;r)  =  — r  F,.(y)<Iy 
(Ph  -  1)  Jo 


(19) 


E{r(.r))  =  ^fE{T{x)\ta  e  [0,M}  +  ^-^-^E{T{x)\i„  € 

ih  th 


Substituting  9  and  13  we  get, 


tr  , ,  _-a.T\  ,  F  f*''  1  f ,,  ^  1  I  .  ^/i 


th  ^  t),  Jf^  (ii,  -  tj-) 


{th  -y)  +  {ph  -  l)thFr,  ( -  -  log  —  j  J  dy 

(^20) 
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V  a  y  J  j 


E  {T{x)}  =  +  I*'-  (p  _  i)F,^  (^.r  -  i  log  dy  (21) 


where 


FrAy) 


Observe  that  the  expression  for  £’{r(;r)}  in  (21)  involves  three  nested  integrals,  which 
makes  it  difficult  to  evaluate  using  a  symbolic  math  package  like  Mathemaiica^^^ ,  After 
some  algebraic  manipulation  the  number  of  nested  integrals  can  be  reduced  to  two.  The 
proof  is  given  in  the  Appendix.  Finally  we  get, 

E{T(X)}  =  +  {p- mA^:)h  -  j{FA.r)  -  e— )  -  ^  £  F{a-  -  z)c-^^dz 
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Lemma  2  Let  Xo  be  defined  such  that  ph{xo)  =  1*  Under  SRPT,  as  th  —)■  oo,  the  rate  at 
which  jobs  complete  during  the  high  load  period  is  XF{xo)  (and  the  rate  of  growth  of  jobs  in 
the  system  is  thus  \F{xo))>  The  p.dj.  of  job  sizes  remaining  in  the  system  is: 


fr{x)  = 


FixoY 


where  x  >  Xq 


Proof:  Consider  the  high  load  period.  Let  W{x^t)  denote  the  work  made  up  by  jobs  of 
of  size  less  than  .t,  arriving  by  time  t,  A  necessary  condition  for  a  job  of  size  x  to  receive 
service  at  time  t  is  that  W{x^  t)  —  t  <  0. 

For  a  job  of  size  x  >  Xo^  the  expected  value  of  W{xfi)  is  p{x)t.  Thus,  by  the  Central 
Limit  Theorem,  W {x^  t)  is  distributed  like  p{x)t+N\/i^  where  N'  is  some  normally  distributed 
random  variable.  As  th  oc  we  see  that  W{x^t)  >  t  with  probability  1.  Thus  the  fraction 
of  jobs  with  size  x  or  greater  which  receive  service  during  the  high  load  period  goes  to  0. 
Thus  the  result  follows. 


Theorem  2  In  an  M/G/l/SRPT  system,  under  the  ON/OFF  load  model,  the  expected 
response  time  for  a  job  of  size  x  is 

lim  IFMsRPT=0^  if 

th 


th-^oo  th 

where  Xq  is  such  that  ph{xo)  =  1* 


lim  if  x>x. 


Proof:  For  jobs  of  size  less  than  clearly  the  response  time  is  independent  of  th.  So 

lim  =  o 

th 

for  X  <  Xo. 

By  Lemma  2,  with  probability  1,  a  job  of  size  greater  than  Xo  receives  service  only  during 
the  OFF  period.  So,  the  expected  waiting  time  of  the  job  during  the  ON  period  will  be 
Moreover,  the  time  since  the  beginning  of  the  OFF  period  until  this  job  is  serviced  is  equal 
to  the  work  made  up  by  jobs  of  size  between  Xo  and  x.  This  amount  of  work  is  equal  to 
fxo  ^^hyfiy)dy,  which  is  equal  to  {ph{x)  -  ph{xo))thi  or  just  {ph{x)  -  1)4,  since  ph{xo)  =  1. 

Thus  it  follows  that  the  mean  response  time  for  a  job  of  size  x,  such  that  x  >  Xq  will  be 

^  +  {ph{x)  -  1)4-  ■ 
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5  Direct  analytic  comparison  of  SRPT  and  PS  under  various 
workloads 


In  Section  4  we  derived  the  expression  for  the  expected  response  time  of  a  job  of  size  x 
under  SRPT  and  PS  for  the  ON/OFF  model.  In  this  section  we  investigate  these  results  for 
specific  job  size  distributions.  We  investigate  2  different  types  of  dist  ributions: 


1.  Exponential. 

2.  Bounded  Pareto.  B{o  =  l.o).  This  is  a  heavy-tailed  distribution. 
Recall  a  Farrfo  distribution  with  j)arameter  o,  is  defined  such  that 


F/-[.V  >  .7‘]  ~  X 


The  5on/ir/cr/-Pf//r/o  distribution  [7]  is  characterized  by  three  parameters:  o,  the  ex¬ 
ponent  of  the  power  law:  h\  the  smallest  possible  job:  and  p.  the  largest  possible  job, 
The  probability  density  function  for  the  Bounded  Pareto  B{k.p.a)  is  defined  as: 


fU) 


xl/ 


0^1 


A*  <  X  <  />. 


In  this  paper,  B{n)  will  denote  the  distribution  B(k\p.a)  ol>tained  by  keeping  the 
mean  fixed  (at  3000)  and  the  maximum  value  fixed  (at  p  =  10^^^).  which  correspond  to 
typical  values  for  Web  workloads  taken  from  [5]. 


5.1  Normalized  Mean  Response  Times  as  a  function  of  job  size 

We  begin  by  focusing  our  discussion  on  two  performance  metrics: 


1.  Mean  response  time  (under  SRPT  versus  under  PS). 

2.  Expected  response  time  for  large  jobs.  Si)ecifically  we  will  be  interested  in  whether 
large  jobs  ‘"starve’'  under  SRPT  scheduling  as  compared  with  PS  scheduling. 


Consider  the  mean  response  time  for  a  job  of  size  ,r,  E  {T(x)}.  Under  our  ON/OFF  load 
model,  E{T'(;r)}  is  proportional  to  //, .  Thus  rather  than  discuss  mean  response  time,  we 
instead  show  normalized  mean  response  time,  defined  as  follows:-^ 


Normalized  Mean  Response  Time  for  job  of  size  x  =  iinif^^^.^, 


E{T(x)  I  ON  period  of  length  f/J 
th 


Figure  1  shows  the  normalized  mean  response  time  as  a  function  of  job  size  under  SRPT 
scheduling  versus  PS  scheduling.  The  job  size  is  expressed  as  a  percentile  of  the  job  size 

^In  the  definition  for  normalized  mean  response  time,  we  assume  that  the  length  of  the  OFF  period  is  oo. 
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Exponential  distribution  with  rho_h  =1.1  Bounded  Pareto  distribution  with  a=1 .5,  rho__h=  1 .1 


Percentile  of  Job  size  Percentile  of  Job  size 


{a)ph  =  1.1 

Exponential  distribution  with  rho_h  =  1.5  Bounded  Pareto  distribution  with  a=1 .5,  rho_h=1 .5 


Percentile  of  Job  size  Percentile  of  Job  size 


{h)ph=l^b 


Exponential  distribution  with  rho_h  =  2 


Bounded  Pareto  distribution  with  a=1 .5,  rho_h=2 


Percentile  of  Job  size  Percentile  of  Job  size 


{c)pk  -  2 


Figure  1:  This  figure  shows  the  normalized  mean  response  times  for  various  values  of  pk 
(ph  =  1-1?  P/j,  =  1*5  and  ph  —  2^.  The  first  cofilmn  assumes  an  exponential  job  size  distribu¬ 
tion  s  and  the  second  column  assumes  the  B(1.5)  distribution. 


distribution  (where  100  percentile  indicates  the  very  largest  job).  Observe  furthermore  that 
due  to  the  choice  of  the  a>axis,  the  area  under  the  PS  (respectively.  SRPT)  curve  corresponds 
to  the  normalized  mean  response  time  under  PS  (respectively,  SRPT).  The  plots  on  the  left 
column  of  Figure  1  considers  the  exponential  job  size  distribution  at  various  values  of  p/,  , 
and  the  plots  on  the  right  consider  the  distribution  B(a  =  1.5). 

We  begin  with  a  few  obser\'ations  which  hold  for  all  plots  and  are  easily  explained: 

Observation  3  Under  SRPT  most  of  the  jobs  hare  a  normalized  response  time  of  0, 

To  see  why,  observe  that  under  SRPT  a  job  of  size  x  will  have  a  normalized  response  time 
of  0,  if  ph(x)  <  1,  that  is  if  the  load  made  up  by  jobs  of  size  <  x  during  the  ON  period  is 
less  than  1.  However  under  PS,  every  job  has  a  non-zero  normalized  response  time. 


Observation  4  Under  SRPT.  let  x  be  the  size  of  the  smallest  job  whieh  has  a  nomzero 
normalized  response  time.  Observe  that  job  x  always  seems  to  have  a  normalized  response 
time  of  exactly 

To  see  why,  observe  at  job  x  ne^Tr  gets  to  run  during  the  ON  period  and  thus  has  an  average 
waiting  time  of  th/2  during  the  ON  period.  Moreover,  job  x  gets  to  run  immediately  when 
the  OFF  period  begins.  Thus  its  normalized  response  time  (as  //,  oc)  is  1/2. 


Observation  5  Under  SRPT,  the  normalized  response  time  approaches  ph  —  \  for  large 
jobs. 


To  see  why,  observe  that  a  large  job  has  a  waiting  time  of  ^  on  the  average  during  the  ON 
period,  and  it  receives  service  only  towards  the  end  during  the  OFF  period.  Since  it  takes 
{ph  —  l)th,  time  to  finish  off  accumulated  work,  the  result  follows. 

We  now  walk  through  Figure  1  and  use  the  above  observations  to  explain  the  plots.  First 
compare  Figure  la  (left)  with  Figure  la  (right).  Both  assume  p/,.  =  1.1,  but  Figure  la(right) 
assumes  an  exponential  job  size  distribution,  while  Figure  la(left)  assumes  a  B(a  =  1.5) 
distribution.  Let  us  contrast  the  curves  with  respect  to  the  performance  of  the  big  jobs. 
Under  the  exponential  job  size  distribution,  jobs  in  the  0-98.5  percentile  have  a  normalized 
response  time  of  0  under  SRPT.  The  job  in  the  98.5^^^  percentile  has  a  normalized  response 
time  of  0.5  under  SRPT,  but  only  0.18  under  PS.  In  fact,  almost  all  the  jobs  in  the  top  1.5 
percentile  do  about  2-3  times  worse  under  SRPT  as  compared  with  PS  under  the  exponential 
job  size  distribution.  By  contrast  under  the  B(1.5)  distribution  ( though  not  very  clear  from 
the  plot)  the  normalized  response  under  SRPT  becomes  non-zero  only  for  jobs  in  the  99.94 
or  higher  percentile.  This  is  true  because  5(1.5)  is  more  heavy  tailed  than  the  exponential 
distribution,  and  thus  only  .06%  of  the  large  jobs  account  for  the  load  between  1  and  1.1. 
Furthermore,  while  a  job  in  the  99.94  percentile  has  a  normalized  response  time  of  0.5  under 
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SRPT,  it  has  a  normalized  response  time  of  almost  0.4  under  PS.  Thus  the  performance  for 
this  job  under  PS  and  SRPT  is  comparable.  In  fact,  for  slightly  bigger  jobs  than  this,  the 
response  times  under  PS  become  even  worse  than  that  under  SRPT. 

Now  contrast  Figure  la(right)  and  Figure  la(left)  with  respect  to  mean  normalized  re¬ 
sponse  time.  For  both  figures  the  area  under  the  SRPT  curve  is  smaller  than  the  area  under 
the  PS  curve,  —  thus  the  mean  response  time  under  SRPT  is  much  lower  than  under  PS. 
The  difference  in  mean  response  time  is  magnified  in  the  case  of  the  J5(1.5)  workload,  be¬ 
cause  almost  all  jobs  have  zero  normalized  time  under  SRPT  (and  all  remaining  jobs  are 
comparable  under  SRPT  and  PS). 

Figure  lb  shows  the  normalized  mean  response  time  under  ph  ~  1.5.  Observe  that 
for  both  the  job  size  distributions  there  is  negligible  starvation  for  large  jobs  under  SRPT 
as  compared  with  PS.  Arguing  as  in  the  above  paragraph,  for  both  job  size  distributions, 
the  mean  normalized  response  time  is  far  lower  under  SRPT  than  under  PS.  Furthermore, 
this  difference  is  exaggerated  in  the  j5(1.5)  distribution  where  under  SRPT  only  2.8%  jobs 
have  a  non-zero  normalized  response  time,  as  opposed  to  about  10%  for  the  exponential 
distribution. 

Finally  when  2  (  Figure  Ic),  we  observe  that  there  is  no  starvation  at  all  under 

SRPT  as  compared  with  PS  under  either  job  size  distribution.  In  fact  every  job  seems  to 
perform  worse  (or  similar)  under  PS,  as  compared  to  SRPT.  This  may  seem  contradictory 
since  one  could  argue  that  at  least  the  very  largest  jobs  should  have  a  larger  response  time 
under  SRPT  than  under  PS.  We  will  explain  this  below. 

We  summarize  the  trends  we  have  seen  in  three  observations.  We  offer  intuition  explain¬ 
ing  each  observation. 

Observation  6  Large  jobs  do  not  necessarily  suffer  under  SRPT  as  compared  with  PS  (as 
is  commonly  believed),  (See  Figure  Ic.)  Particularly  for  heavy-tailed  distributions^  large  jobs 
often  do  at  least  as  well  under  SRPT  as  compared  with  PS^  with  respect  to  their  expected 
response  time.  (See  Figure  1  whole  right  column). 


To  see  why  this  is  the  case,  observe  that  although  large  jobs  do  badly  under  SRPT,  they 
do  almost  equally  badly  under  PS.  The  point  is  that  the  average  amount  of  service  received 
by  a  large  job  during  an  ON  period  is  negligible  compared  to  its  size.  Thus  this  job  stays 
in  the  system  throughout  the  ON  period  (since  its  arrival).  Moreover  it  is  among  the  last 
of  the  jobs  to  complete  during  the  OFF  period,  since  its  remaining  size  at  the  beginning  of 
the  OFF  period  is  large  compared  to  other  remaining  jobs. 


Observation  7  For  a  fixed  ph,  the  more  heavy-tailed  the  job  size  distribution  the  better  the 
performance  of  big  jobs  under  SRPT  as  compared  with  PS.  Also,  the  more  heavy-tailed  the 
job  size  distribution  the  greater  the  improvement  in  the  normalized  mean  response  time  under 
SRPT  as  compared  with  PS.  (See  Figure  1  right  column  as  compared  with  left  column). 
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To  see  why  this  is  the  case,  recall  that  in  a  more  heavy-tailed  distribution,  there  are  fewer 
jobs  comprising  the  excess  load.  These  few  jobs  are  very  big  (think  of  them  as  elephants). 
Now,  by  the  same  argument  as  above,  these  elephants  don’t  make  much  progress  under  PS 
during  the  ON  period,  thus  their  response  times  are  comparable  under  PS  and  under  SRPT. 
Furthermore,  all  jobs  other  than  the  elephants  experience  zero  normalized  mean  response 
time  under  SRPT.  Thus  the  overall  normalized  mean  response  time  under  SRPT  is  much 
lower  than  under  PS. 

Observation  8  The  higher  the  ralve  of  the  lower  the  starvation  of  big  jobs  under  SRPT 
as  compared  with  PS.  (See  Figure  1) 


This  is  a  surprising  observation,  in  that  one  might  conjecture  that  a  low  value  of  pr 
(barely  overloaded  ON  times)  would  yield  less  starvation  of  big  jobs.  However  it  turns  out 
that  the  reverse  is  true.  Here's  some  intuition:  When  />/,  is  closer  to  1,  PS  gets  more  work 
done  on  the  big  jobs  during  the  ON  period.  Thus  at  the  start  of  the  OFF  period,  the 
response  times  of  the  big  jobs  under  PS  is  much  smaller.  Thus  the  big  jobs  in  SRPT  appear 
to  be  ’’starved"  by  comparison,  although  in  fact  their  response  times  have  improved  as  well 
—  but  not  by  as  much.  This  intuition  is  in  complete  agreement  with  Figure  1. 


5.2  Growth  in  number  of  jobs  in  the  system  over  time 


We  now  consider  the  number  of  jobs  in  the  system  as  a  function  of  time  under  PS  and  SRPT. 
The  number  in  system  is  an  interesting  practical  metric.  Consider  as  an  example  a  Web 
server  which  services  its  requests  in  SRPT  order,  as  opposed  to  the  traditional  PS  service 
order.  The  number  of  requests  in  the  system  corresponds  to  the  number  of  simultaneously 
open  connections  in  the  Web  server.  The  greater  this  number  the  more  overhead  is  required 
by  the  Web  server.  Furthermore,  if  this  number  gets  too  high,  the  Web  server  sim]>ly  crashes. 

The  mean  number  of  jobs  in  the  system  is  obviously  an  increasing  function  of  //, .  Thus 
we  instead  look  at  the  fraction  of  arrivals  remaining  in  the  systenh  which  is  defined  as: 


Fraction  of  arrivals  remaining  —  linii^^^^, 


E{N(t)  I  ON  period  of  length  tr} 


where  N{t)  denotes  the  number  of  jobs  in  the  system  at  time  t.  We  consider  a  normalized 
time  axis,  showing  t/th^  rather  than  t. 

Figure  2  shows  the  fraction  of  arrivals  remaining  as  a  function  of  normalized  time,  for 
various  values  of  pr,  under  SRPT  and  PS  scheduling.  Observe,  a  value  of  1  on  the  x-axis 
indicates  the  end  of  an  ON  period.  The  plots  in  the  left  column  assume  an  exponential 
job  size  distribution,  whereas  the  plots  in  the  right  column  assume  the  distribution  5(1.5). 
Again^  the  area  under  the  curves  is  proportiemal  to  the  mean  n  umber  of  jobs  in  the  systein 
(hence  to  the  mean  response  time). 
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Exponential  distribution,  rho_h=1 .1  Bounded  Pareto  distribution  with  a=1 .5,  rho_h  =  1.1 


{a)ph  =  1-1 


Exponentiai  distribution  with  rho_h  =  1.5  Bounded  Pareto  distribution  with  a=1.5,  rho_h  =  1.5 


{b)pi,  =  1.5 

Exponentiai  distribution,  rho_h=2  Bounded  Pareto  distribution  with  a=1 .5,  rho_h  =  2 


{c)ph  =  2 


Figure  2:  This  figure  shows  the  fraction  of  arrivals  remaining  as  a  function  of  normalized 
time  under  (a)  ph  —  l.l^  (b)  ph  ~  1.5  and  (cj_^h  =  2.  The  left  column  corresponds  assumes 
an  exponential  job  size  distribution^  and  the  right  column  a  heavy-tailed  B(1.5)  distribution. 


Below  we  state  a  few  observations  about  the  graphs  and  provide  intuition  explaining 
these  observations. 


Observation  9  For  all  plots  in  Figure  2.  the  ninubcr  of  jobs  in  the  system  under  SEPT  is 
cdweiys  significantly  less  than  that  under  PS, 


Observation  10  For  a  fixed  />/, .  the  more  heary-iailed  the  job  size  distribufiern  the  greater 
the  improvement  in  number  ejf  jejbs  in  the  system  under  SEPT  as  compared  ivith  PS.  (See 
Figure  2  right  column  as  compared  with  left  cejiumn). 


For  example,  Figure  2a  (left)  shows  the  number  in  system  for  the  exponential  distribution 
when  ph  =  1.1.  Observe  that  the  number  in  system  under  SRPT  is  always  4-5  times  lesser 
than  that  under  PS.  Figure  2a  (right)  shows  the  number  is  system  for  fi(1.5)  distribution 
when  /)/,  1.1.  The  improvements  in  the  mean  number  in  system  under  SRPT  are  much 

more  significant,  about  15-20  times  better  than  that  under  PS.  This  observation  coupled  with 
Observation  6  about  starvation  in  Figure  la  (right)  makes  a  very  strong  case  for  SRPT.  Not 
only  is  there  a  15-20  times  improvement  in  the  mean  response  time  under  SEPT,  but  this 
improvement  dejes  not  cerme  at  the  cost  of  starving  large  jejbs. 

Observation  9  makes  sense  since  SRPT  obviously  minimizes  the  number  of  jobs  in  the 
system  at  any  time.  The  intuition  behind  Observation  10  is  the  same  as  that  behind  Obser¬ 
vation  7. 

Observation  11  Thejugh  still  significant,  the  releitive  advantage  of  SEPT  ejver  PS  (ivith 
respect  to  number  of  jobs  in  the  system)  decreases  at  higher  values  of  pj^.  (See  Figures  2b 
and  2cj. 


Observe  that  this  does  not  contradict  Observation  8,  since  that  observation  is  concerned 
only  with  the  large  jobs. 

Observation  12  The  number  in  system  increases  linearly  during  the  ON  periejd  for  both 
SEPT  and  PS. 


This  follows  from  Lemma  1  and  Lemma  2. 


Observation  13  The  curve  for  the  n  umber  of  jobs  under  SEPT  during  the  OFF  periejd  is 
always  convex. 


This  follows  since  SRPT  works  on  jobs  with  the  smallest  remaining  size  first,  thus  the  rate 
of  clearance  of  jobs  is  maximum  in  the  beginning  of  the  OFF  period  and  then  decreases. 
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Observation  14  Observation  13  above  is  not  true  for  PS.  In  fact,  since  PS  timeshares 
among  all  the  jobs,  it  somewhat  delays  getting  jobs  out  at  the  beginning  of  the  OFF  period. 
This  can  be  observed  significantly  in  Figure  2c  (right).  Thus  SRPT  not  only  accumulates 
fewer  jobs,  but  it  also  gets  them  out  as  quickly  as  possible. 


We  end  with  an  interesting  side  note: 

Observation  15  The  number  in  sysfenr  under  PS  decreases  linearly  under  PS  for  the  ex¬ 
ponential  distribution. 


To  see  this,  observe  that  at  the  end  of  an  ON  period,  the  remaining  sizes  of  the  jobs  are  also 
exponential!}^  distributed  with  some  rale,  say  //, (since  the  distribution  of  the  remaining  size 
of  a  job  conditional  on  the  fact  t  hat  it  has  received  x  amount  of  service  is  still  exponential). 
Now  observe  that  regardless  of  how  many  jobs  there  are  remaining  at  the  end  of  the  ON 
period,  the  rate  at  which  jobs  will  compete  under  PS  is  a  constant:  //,.  (This  follows,  since 
if  there  are  n  jobs  in  the  system,  then  since  each  job  receives  of  the  service,  it  is  likely 
to  finish  with  rate  Since  there  are  n  jobs,  the  total  rate  at  which  jobs  leave  the  system 
will  be  //.) 


6  Conclusion 


This  paper  examines  a  load  model  consisting  of  alternating  ON/OFF  periods  where  the 
system  load  exceeds  1  during  the  ON  period  and  is  0  during  the  OFF  period.  Under  this 
load  model,  we  compare  SRPT  and  PS  scheduling.  We  find  that  SRPT  scheduling  is  a  big 
win  with  respect  to.  mean  performance  metrics  like  mean  response  time  and  mean  number  of 
jobs  in  the  system.  More  surprisingly,  we  find  that,  in  the  case  where  the  job  size  distribution 
is  heavy-tailed,  this  win  does  not  come  at  the  cost  of  starving  the  large  jobs.  In  fact  all  jobs 
including  the  very  largest  perform  better  or  only  marginally  worse  under  SRPT  scheduling, 
as  compared  with  PS  scheduling. 

The  above  results  seem  counterintuitive.  Shouldn’t  large  jobs  starve  more  under  SRPT 
given  temporary  overload?  We’ve  found  that  the  answer  is  no,  but  the  reason  can  only 
partially  be  attributed  to  the  superior  properties  of  SRPT.  At  least  equally  relevant  is  the 
poor  performance  of  PS.  Our  analysis  shows  that  PS  is  particularly  ineffective  in  dealing 
with  periods  of  temporary  overload.  Due  to  its  time-sharing  nature,  it  deteriorates  the 
performance  of  all  the  jobs.  Moreover,  PS  is  particularly  slow  at  getting  the  system  “back 
to  normal”  once  the  overload  has  disappeared.  By  contrast,  SRPT  accumulates  far  fewer 
jobs  during  the  overload  period,  and  is  also  much  more  efficient  at  getting  them  out  once  the 
overload  period  is  over.  A  heavy-tailed  job  size  distribution  works  strongly  in  SRPT’s  favor 
because  it  allows  SRPT  to  complete  all  but  a  small  fraction  of  the  jobs  during  the  overload 
period. 
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The  superior  performance  of  SRPT  under  fluctuating  load  conditions  and  heavy-tailed 
job  sizes  make  SRPT  an  attractive  alternative  to  the  more  traditional  PS  scheduling  in 
applications  where  job  sizes  are  known,  or  can  be  estimated. 
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7  Appendix 


We  will  simplify  the  integral  I  =  //j'  Fr,  ^  log  dy 
Set  y  =  then  dy  =  -athe~‘'~dz  and 


I  = 


z)athe  °'dz 
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Applying  integration  by  parts,  we  get 

I  =  [-F,^{x  -  z)the--%  -  j\he-<^^Tr{x  -  z)^^dz 
I  -  Fr^{x)th  -  h 


where 


h  =  l%he-^^Fr{x-z)-^dz 


E[Sr] 


We  will  show  in  Lemma.  4  that 

th 


h  = 


Thus  giving  us 


{2a)E[Sr] 


{Fr{x)  -  e' 


^  2aE[Sr]  i 


I  =  FrA^)tk  -  TFZ^AFAx)  - 


(2a)£[,S,]' 


2aE[Sr] 


r  F{x  -  z)€-^^dz 
Jo 


(22) 


We  first  begin  with  an  identity  which  relates  the  pdf  and  the  cdf  of  the  remaining  service 
times  of  jobs  under  PS. 

Lemma  3  Let  fr{y)  and  Fr{y)  denote  respectively  the  pdf  and  cdf  of  the  remaining  service 
of  the  jobs  at  the  start  of  the  OFF  period  under  PS.  Then 

AF(y)  fAy) 


Fv{y)  = 


a  a 


Proof:  By  (16)  we  know  that 


Fr{y)  = 


fA'>^f{z){l-e-<^-y))dz 


AF(y) 


POO  \ 

/  -f{z)e-^^dz 

Jy  Oj 


Since  fr{y)  =  -j^Fr{y) 

—A  A  A 

-My)  =  —f{y)  +  e-y-f{y)e-‘^y  -  ae'^y  /  -f{z)e--^~dz 

a  CL  Jy  ^ 

Observing  that  the  first  and  second  terms  cancel  out,  we  get 

POO 

My)  =  Xe^y  /  f{z)e-‘^^dz  (24) 

Jy 

Comparing  the  expressions  for  Fr{y)  and  My)  given  by  (23)  and  (24),  the  result  follows. 
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Lemma  4  Let  /j  be  defined  as  in  equation  22.  then 


(•2«)i?[,S',] 


Proof: 

fih  ,- 
E[ 

th 


ih 


fh 


-ii.v\  ,  li 


(F,.{x)-e-"'-)  +  : 


'  '  '  ■2«/-:[,s',] 

The  last,  equality  follows,  since  tlu'  final  term  in  this  expression  is  just  I\.  Thus 

^  l‘  i~r  /  V  -ri.lv  . 


u  = 


(2«)£’[.S',] 


’■\  I  r  T( 

>  +  2^CT5J./o 
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